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Adaptive  Mathematical  Morphology  for  Range  Imagery 


Jucqucs  G.  Vcriy  and  Richard  L.  Delanoy 


Abstract — We  explore  the  application  of  adaptive  (i.e.,  data-dependeni) 
mathematical  morphology  techniques  to  range  imagery,  i.e.,  the  use  of 
structuring  elements  that  automatically  adjust  to  the  grayscale  values  in 
a  range  image  in  order  to  deal  with  le.g.,  extract  or  eliminate)  features 
of  known  physical  sizes. 


I.  Introduction 

Often,  the  output  of  a  mathematical-morphology  (MM)  operation 
[1]  (either  binary  or  grayscale)  describes  how  well  a  shape,  called 
a  structuring  element  (SE),  either  fits  or  does  not  fit  inside  a  local 
image  feature. 

Whereas  the  SE’s  used  in  most  applications  remain  constant  as 
they  probe  an  image,  there  are  situations  where  SE’s  must  change 
their  size,  orientation,  and/or  shape  during  probing.  If  these  changes 
are  determined  a  priori,  independently  of  the  data,  the  SE’s  can  be 
said  to  be  space  variant.  If  the  change  is  made  on  the  basis  of  the 
value(s)  of  one  or  more  pixels  around  the  pixel  being  probed,  the 
SE’s  can  be  said  to  be  data  dependent  or  adaptive.  We  are  aware  of 
only  a  few  papers  dealing  with  space-variant  SE’s,  e.g.,  [2]  (also  in 
[3])  and  (4],  and  of  none  dealing  with  adaptive  SE’s. 

Since  range  imagery  [S]  contains  significant  shape  information  and 
since  MM  deals  with  shape  in  images,  it  is  also  surprising  that  so  few 
papers  have  been  written  on  the  application  of  MM  to  range  imagery 
(examples  are  [6]-[10|). 

Since  “angle-angle”  range  images  are  subject  to  the  usual  perspec¬ 
tive  distortions,  the  apparent  length  of  a  feature  in  such  images  is 
a  function  of  the  feature  range.  With  range  available  at  each  pixel, 
one  can  simultaneously  process  (e.g.,  extract  or  eliminate)  all  the 
differently-scaled  instances  of  the  feature  or  object  of  interest  by 
adapting  the  sizes  of  the  SE(s)  to  the  local  range. 

For  simplicity,  we  will  assume  that  all  SE’s  of  interest  become  fully 
determined  when  their  scalable  x  size  and  y  size  are  specified  (x  and 
y,  respectively,  refer  to  the  horizontal  and  vertical  image  axes).  In 
other  words,  we  assume  that  adapting  an  SE  to  range  simply  amounts 
to  finding  the  appropriate  x  and  y  sizes.  Examples  of  useful  SE’s  that 
have  such  properties  are  the  rectangular  and  ellipsoidal  SE’s  (either 
binary  or  grayscale).  Note  that,  for  this  class  of  SE’s,  changing  the  x 
and  y  sizes  can  in  some  cases  cause  a  spatial  compression/expansion 
of  the  pattern  (either  binary  or  grayscale).  The  goal  of  this  paper 
is  to  develop  a  systematic,  practical  methodology  for  designing  and 
applying  adaptive  SE’s  for  range  imagery  under  the  constraints  just 
described. 


II.  Principles 

Let  us  consider  a  range  image  i?  of  a  scene  5.  The  grayscale  value 
r,  associated  with  each  pixel  is  related  to  the  distance  r„,  (say,  in 
meters)  between  the  corresponding  patch  of  5  and  the  sensor  by 
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Fig.  1.  Range  intervals  associated  with  integer-length  SE's  when  the  SE's 
must  fit  inside  a  feature. 


where  ro  is  a  range  offset  corresponding  to  the  grayscale  value  of  0, 
6  is  the  range  quantization  value,  and  [x]  is  the  integer  part  of  x. 

The  length  in  units  of  pixels  of  a  horizontal  feature  F  (in 
If)  that  b'>s  an  actual  length  of  x,„  meters  (in  S)  and  is  located  at 
a  distance  of  r,„  meters  from  the  imaging  sensor  is  given  by  the 
small-angle  approximation 


where  Oj.  is  the  horizontal  angular  width  (in  radians)  of  a  pixel. 
This  expression  can  also  be  used  to  find  the  x  dimension  rij^  (in 
pixels)  of  the  SE  required  to  detect  F  in  B  (for  convenience,  we  can 
temporarily  focus  on  the  case  of  horizontal,  rectangular,  1-pixel  high 
SE’s).  Notice  that  we  use  primed  symbols  like  n'^  to  denote  arbitrary 
lengths  in  units  of  pixels,  and  unprimed  symbols  like  rij.  for  lengths 
that  are  an  integer  number  jf  pixels. 

From  the  functional  dependency  of  Cn,  upon  ri'^  for  fixed  and 
x,„  (e.g.,  as  in  Fig.  1),  we  observe  the  following. 

First,  consider  SE’s  that  must  fit  inside  F  in  B.  Such  SE’s  will  be 
said  to  be  of  type  MF.  For  example,  a  SE  used  in  an  opening  with 
the  intent  of  preserving  a  protrusion  must  be  of  type  .V/F;  ditto  for 
closing  and  intrusion.  A  fixed-length  SE  designed  to  fit  inside  F  at 
a  given  range  will  fit  in  the  same  F  at  a  shorter  range,  but  not  at 
a  longer  range.  Since  the  localization  of  F  based  on  a  fixed-length 
SE  becomes  less  sharp  as  the  range  decreases,  one  should  switch  to 
the  next  larger  size  SE  as  soon  as  its  design  range  is  reached.  In 
summary,  in  the  case  of  .\fF-type  SE’s,  the  range  interval  where  a 
given  SE  can  be  used  is  from  (and  including)  its  associated  range  (i.e., 
the  range  at  which  it  fits  exactly  in  F)  down  to  (but  excluding)  the 
design  range  of  the  next  larger  SE.  For  example,  when  all  integer- 
length  SE’s  are  available,  the  semi-open  range  interval  for  the  SE 
corresponding  to  nr  =  1  is  obtained  from  the  solid  curve  segment 
labeled  1  in  Fig.  1  (  the  interval  is  shown  along  the  r„-axis).  Pixels 
with  ranges  corresponding  to  Hi  =  0  should  not  be  processed  at  all. 
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Fig.  2.  Range  intervals  associated  with  integer-length  SE's  when  the  SE’s 
must  not  fit  inside  feature. 


2.  For  SE’s  of  type  .M\F: 

tir  .ni.n  =  Smallest  integer  in  .V' 

strictly  greater  than  ri'^  „„„ 

"t  max  =  smallest  integer  in  .V' 

strictly  greater  than  ii'x,,„ax  ■ 

Thus  the  set  of  SE’s  that  can  be  called  upon  is  the  ordered  subset 

.\'r  —  {ur.i  I  E  =  1.  h\\  tir.k  €  -V’ : 

rtr.niin  ^  Ur.lt  ^  Ux.niaxtUx.L  ^  nx.L-*-l} 

of  N’,  where  A'x  <  A'*  is  the  maximum  index  k  needed, 

B.  Selected-SEs  ’  Range  Intervals  in  Meters 

Denoting  the  actual  range  (in  meters)  associated  with  each  n^.k 
in  Sr  by  Tr.k.m,  with 

Xm 

rr.k.m  =  - 

nx.LOx 

it  can  be  shown  that  the  lower  and  upper  limits  (in  meters)  of  the 
semi-open  interval  [rx,/.,*.tn.  fx.H.t.m]  are  as  follows. 

1)  For  SE’s  of  type  MF : 


Second,  consider  SE’s  that  must  not  fit  inside  F  in  R.  Such  SE’s 
will  be  said  to  be  of  type  MSF.  For  example,  a  SE  used  in  an 
opening  with  the  intent  of  eliminating  a  protrusion  must  be  of  type 
MSF;  ditto  for  closing  and  intrusion.  Here,  the  range  interval  where 
a  given  SE  can  be  used  is  from  (but  excluding)  its  associated  range 
(i.e.,  the  range  at  which  it  fits  exactly  in  F)  up  to  and  including  the 
design  range  of  the  next  smaller  SE  (Fig.  2). 

in  practice,  it  may  not  be  feasible  to  use  all  the  sizes  tir  the:  arc 
necessary  to  cover  the  total  range  interval  of  R.  The  dashed  lines  in 
Figs.  1  and  2  show  the  range  intervals  and  the  corresponding  SE  sizes 
for  the  hypothetical  case  where  only  the  odd  sizes  tir  are  available. 


III.  Implementation 

Denote  the  minimum  and  maximum  values  of  R  by  r,4.,  and  rs.i, 
where  i  is  a  reminder  that  these  numbers  are  integers.  Using  (1),  one 
can  find  the  limits  of  the  span  [PA.m.  fB.ml  of  actual  range  values 
(in  meters)  in  R,  i.e., 

{PA.m  =  To  +  TA.ift 

TB.m  =  To  +  (r/3,,  -f  1)6. 


A.  Selection  of  Necessary  SE’s 

Using  (2),  one  finds  that  the  required  interval  of  sizes  n'^  is 

[^x.min  '  Tix.max]  '  with 


^x.min  —  ^  Tlx, max 


I*  A  ,m^^i 


Assuming  that  the  only  SE  sizes  available  are  those  in  the  ordered 
set 


^'x  =  {tii.k  |E  =  l,A’*:nx,i  =0;nr.k  <  Ux.l-i-i} 


where  K'r  is  the  number  of  available  sizes,  the  corresponding  interval 
of  integers  tir  is  [ni.min>ni,max]  where  the  limits  are  as  follows. 

1.  For  SE’s  of  type  MF: 

tlx. min  =  largest  integer  in  S^ 

less  than  or  equal  to  v’, 
nr, max  =  largest  integer  in  S^ 

less  than  or  equal  to  n'r, max  ■ 


_  j  rr.k+l.m  if  k  <  Kr 
\  TA.m  if  A-  =  A'x 


.k  .m  —  I*  X  ,  .  n 


2)  For  SE’s  of  type  MSF: 


t'r,I..k,m  —  rr.k.n 


f  Tj.jt— l.m  if  A'  1 

Tx.H.fc.m  —  A  I.  _  1 

(  rff  m  ll  k  —  k  . 


C.  Selected-SEs  ’  Range  Intervals  in  Range  Integers 

Expressing  interval  limits  in  terms  of  range  integers  (instead  of 
meters)  can  speed  up  the  repeated  comparison  of  pixel  grayscale 
levels  to  the  various  intervals.  Introducing 


I  I*  x.it.m  1*0 

Tx.L..  -  I 

one  can  show  that  the  upper  and  lower  limits  (in  range  integers)  of 
the  closed  interval  [tx.t.l.,.  Tx.h.l,,]  associated  with  each  size  Ur.k 
in  Sr  are  as  follows.' 

1)  For  SE’s  of  Type  MF: 

Tr.H.k.,  =  largest  integer  smaller  than  or  equal  to  Txjt,, 
f  Tx.h.h-i.i  +  1  if  A  <  Ax 

Tx./..*.,  -  |t^.  if  a  =  Ay. 

2)  For  SE’s  of  type  MSF: 

rr,i..k.x  =  smallest  integer  strictly  greater  than  Tx,*,. 


f  Tr.L,k-\.x  -  1  if  A  >  1 
ifA  =  l. 


D.  General  Implementation  Strategy 

So  far,  we  have  focused  on  the  x  dimension  of  an  SE  and  on 
the  particular  case  of  a  1 -pixel  high  horizontal  SE.  Of  course,  the 
argument  can  be  identically  repeated  for  the  y  dimension  and  for  a 

'The  limits  Tr,i,k.,  and  Tx.b.l.,  are  derived  independently  of  their 
counterparts  Tr.u.k.m  and  rr,H.k.m- 


(Cl  cli'slni;  /’ 


(1)1  iHll|i|ll  inMCf 


I  iu.  .)  I  ',lr:ii.-|((<n  nl  I. ink  Null^.■^  Ironi  :i  v\ nltii.lii.'  r.insic  iiii.im.-:  (A)  r.inuc  im.mc  II.  (Bi  ilnsini; 

II  -r  -  I!''  ■  ^ 


1 -pixel  wide  \ertie;il  SI  .  In  (he  e.ise  iit  idi  ;irhilnir\  SI-.  one  niiisl 
deni  w  ith  both  dimensions  sirnnll;meoiis|s . 

rile  uenernl  siraleey  for  eftieienlly  iniplemenlinj:  .idaplixe  MM 
algorithms  is  to  (a)  determine  the  sets  \  ,  and  A,,  of  applieahle  SI- 
si/es.  (  b)  liiiil  the  interval  /,  i  ;  /  i  of  range  \ aloes  (either  in  meters 

or  in  range  integers)  eaeh  Sb.  si/e  n ,  <  c  A  ,  ( n ,,  t  A  .,  i  applies  to, 
(e)  denote  by  /j  ,  the  (possibly  empt\  )  range  interval  that  is  e(>mmon 
to  /,  1  atid  1,1  i  and  assoeiate  with  it  the  available  Sb  with  si/es 
n ,  t  and  II,,  I.  and  (d)  perform  the  desired  operation  on  the  imatie  of 
interest  by  seleeting  the  (tppropriate  av  .iilable  SI  at  eaeh  pixel  based 
upon  the  range  at  that  pixel,  bins  last  step  mav  be  implemented  in 
a  variety  of  ways,  ineiinling  in  parallel,  e  g.,  with  one  processor  per 
applicable  Sb.  Once  ((gain,  one  should  stress  that  the  processing  will 
depend  upon  the  type  of  each  SI:  of  interest,  i.e..  Ml'  or  1/\7'. 

IV.  Al’I'l  K  At  ION 

Adaptive  MM  is  routinely  used  bv  the  authors  for  the  extraction 
of  vehicles  of  known  si/es  from  rciil  low -resolution  (2 -b  meters) 
tactical  laser-radar  imagery  .  It  is  one  of  the  lechnii|iies  used  in  the 
■XI'RS  system  described  in  [III.  Here,  to  better  demonstrate  the 


potential  of  adaptive  MM.  we  use  a  2-m  resolution  smiiIh  Ih  image 
ol  a  scene  eonsisting  ol  a  ground  plane,  a  baekgroiind  vertieal  vvall 
at  1. 71)1)  111,  and  ob|ecls;  a  ni-wide  lank  and  a  2  4  ni-wide  box. 
both  at  711(1  111.  and  a  replied  of  the  tirst  lank  at  l..x()(l  in  (big  .'(.A)). 

In  images  taken  roughly  at  ground  level,  vehicles  that  have  width  ii 
and  length  /  produce  observ  iible  silhouettes  w  hose  length  should  be  in 
the  approximate  interval  ' u  .  n  -  +  C  .  b'or  the  lank  in  big.  .‘'(A), 
the  limits  ol  this  interviil  for  the  lank  bodv  are  approximatelv  ,7  m 
and  7  ni. 

An  im.ige  I!  th.il  highlights  the  tank  bodies  in  the  image  II  of 
big.  .b(.A)  can  be  created  by  using  the  Jib/i’rc/K c-of-c  /mi/igv 

I!  ^  '  -  It” 

where  //-  is  a  4//  -type  ad.iplive  Sb  designed  for  a  leature  length 
.1- ,.  —  7.1)  111  and  // ;  is  a  .\/.\/  -type  adaptive  .Sb  designed  for 
.I-.,,  —  .'',()  Ill  (both  Sb's  are  1-pixel  high).  I'lie  result  is  shown  in 
big.  .'([)).  birst.  note  that  both  lank  bodies  are  correctly  highlighted 
in  spile  of  their  different  apparent  widths. 

bliis  results  from  the  fact  that  the  v  isible  .7  (i  in  phvsical  widths  of 
the  bodies  are  w  ithin  the  allowed  .'  -7  in  phvsical-si/e  range  for  the 
visible  widths  of  such  bodies.  Second,  note  that  the  box  in  front  of  the 


IBHE  TRANSACTIONS  ON  IMAOE  PROC  ESSING.  VOL.  NO.  2.  APRIL  IW 


275 


farthest  tank  is  nut  detected  even  though  its  apparent  width  is  between 
those  of  the  tank  bodies.  Of  course,  the  reason  is  that  the  physical 
width  of  the  box  is  outside  the  allowed  range  for  physical  body  sizes. 
Clearly,  the  exact  same  technique  would  extract  any  number  of  tanks 
(e.g.,  in  column),  provided  the  tanks  do  not  significantly  overlap. 

The  success  of  our  simple  object  extraction  procedure  is  not  only 
due  to  the  well-known  “sieving”  properties  of  successive  closings 
(or  openings)  with  differently  sized  SE’s  (Ij,  but  also  to  the  use  of 
range-adaptive  SE’s  properly  designed  with  respect  to  the  MF/M.\F 
dichotomy. 

V.  Conclusions 

In  this  work,  we  have  pointed  out  the  role  of  adaptive  MM  in  the 
processing  of  range  imagery.  We  have  emphasized  the  importance  of 
correctly  distinguishing  between,  and  implementing,  adaptive  (i.e., 
data  dependent)  SE’s  that  either  “must  fit”  or  “must  not  fit”  within 
the  feature(s)  of  interest.  It  should  be  clear  that  the  choice  among  the 
elements  of  the  MF/M.\F  dichotomy  is  based  on  the  problem  at 
hand,  and  is  not  an  intrinsic  property  of  the  MM  operation  used.  In 
other  words,  there  is  no  such  conclusion  as  saying  that  MM  operation 
X  (e.g.,  dilation,  erosion,  opening,  closing,  etc.)  should  always  be 
used  with  an  adaptive  SE  designed  so  that  it  fits  (or,  similarly,  does 
not  fit)  inside  the  feature  of  interest. 

Although  we  have  discussed  adaptive  MM  in  the  specific  case  of 
range  imagery,  this  technique  is  applicable  to  any  type  of  image  for 
which  the  distance  to  a  scene  element  is  available  for  each  pixel.  Such 
a  situation  occurs  in  laser-radar  imaging  where  pixel-registered  range 
and  intensity  images  are  often  available.  In  this  particular  application, 
adaptive  MM  can  be  applied  to  the  intensity  image  by  using  the 
distance  information  available  from  the  range  image. 

The  techniques  presented  are  limited  to  the  case  where  the  SE’s  can 
be  scaled  through  their  x  and  y  sizes.  Clearly,  the  opportunity  exist 
for  developing  a  more  general  theory  and  methodology  for  designing 
and  applying  adaptive  SE’s  to  range  imagery.  Furthermore,  the  whole 
question  of  how  to  efficiently  implement  adaptive  MM  should  be 
investigated  in  detail. 
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